LIDAR-Camera Fusion for Road Detection Using Fully Convolutional Neural Networks

In this work, a deep learning approach has been developed to carry out road detection by fusing LIDAR point clouds and camera images. An unstructured and sparse point cloud is first projected onto the camera image plane and then upsampled to obtain a set of dense 2D images encoding spatial information. Several fully convolutional neural networks (FCNs) are then trained to carry out road detection, either by using data from a single sensor, or by using three fusion strategies: early, late, and the newly proposed cross fusion. Whereas in the former two fusion approaches, the integration of multimodal information is carried out at a predefined depth level, the cross fusion FCN is designed to directly learn from data where to integrate information; this is accomplished by using trainable cross connections between the LIDAR and the camera processing branches. To further highlight the benefits of using a multimodal system for road detection, a data set consisting of visually challenging scenes was extracted from driving sequences of the KITTI raw data set. It was then demonstrated that, as expected, a purely camera-based FCN severely underperforms on this data set. A multimodal system, on the other hand, is still able to provide high accuracy. Finally, the proposed cross fusion FCN was evaluated on the KITTI road benchmark where it achieved excellent performance, with a MaxF score of 96.03%, ranking it among the top-performing approaches

LIDAR-based Driving Path Generation Using Fully Convolutional Neural Networks

In this work, a novel learning-based approach has been developed to generate driving paths by integrating LIDAR point clouds, GPS-IMU information, and Google driving directions. The system is based on a fully convolutional neural network that jointly learns to carry out perception and path generation from real-world driving sequences and that is trained using automatically generated training examples. Several combinations of input data were tested in order to assess the performance gain provided by specific information modalities. The fully convolutional neural network trained using all the available sensors together with driving directions achieved the best MaxF score of 88.13% when considering a region of interest of 60x60 meters. By considering a smaller region of interest, the agreement between predicted paths and ground-truth increased to 92.60%. The positive results obtained in this work indicate that the proposed system may help fill the gap between low-level scene parsing and behavior-reflex approaches by generating outputs that are close to vehicle control and at the same time human-interpretable.Comment: Changed title, formerly "Simultaneous Perception and Path Generation Using Fully Convolutional Neural Networks

Fast LIDAR-based Road Detection Using Fully Convolutional Neural Networks

In this work, a deep learning approach has been developed to carry out road detection using only LIDAR data. Starting from an unstructured point cloud, top-view images encoding several basic statistics such as mean elevation and density are generated. By considering a top-view representation, road detection is reduced to a single-scale problem that can be addressed with a simple and fast fully convolutional neural network (FCN). The FCN is specifically designed for the task of pixel-wise semantic segmentation by combining a large receptive field with high-resolution feature maps. The proposed system achieved excellent performance and it is among the top-performing algorithms on the KITTI road benchmark. Its fast inference makes it particularly suitable for real-time applications

Deep Learning Applications for Autonomous Driving

This thesis investigates the usefulness of deep learning methods for solving two important tasks in the field of driving automation: (i) Road detection, and (ii) driving path generation. Road detection was approached using two strategies: The first one considered a bird\u27s-eye view of the driving scene obtained from LIDAR data, whereas the second carried out camera-LIDAR fusion in the camera perspective. In both cases, road detection was performed using fully convolutional neural networks (FCNs). These two approaches were evaluated on the KITTI road benchmark and achieved state-of-the-art performance, with MaxF scores of 94.07% and 96.03%, respectively. Driving path generation was accomplished with an FCN that integrated LIDAR top-views with GPS-IMU data and driving directions. This system was designed to simultaneously carry out perception and planning using as training data real driving sequences that were annotated automatically. By testing several combinations of input data, it was shown that the FCN having access to all the available sensors and the driving directions obtained the best overall accuracy with a MaxF score of 88.13%, about 4.7 percentage points better than the FCN that could use only LIDAR data

Lidar–camera semi-supervised learning for semantic segmentation

In this work, we investigated two issues: (1) How the fusion of lidar and camera data can improve semantic segmentation performance compared with the individual sensor modalities in a supervised learning context; and (2) How fusion can also be leveraged for semi-supervised learning in order to further improve performance and to adapt to new domains without requiring any additional labelled data. A comparative study was carried out by providing an experimental evaluation on networks trained in different setups using various scenarios from sunny days to rainy night scenes. The networks were tested for challenging, and less common, scenarios where cameras or lidars individually would not provide a reliable prediction. Our results suggest that semi-supervised learning and fusion techniques increase the overall performance of the network in challenging scenarios using less data annotations

Intronic CYP46 polymorphism along with ApoE genotype in sporadic Alzheimer Disease: from risk factors to disease modulators

Increasing biological and clinical findings argue for a link between brain cholesterol turnover and Alzheimer Disease (AD), high cerebral levels of the former increasing Abeta load. Cerebral cholesterol elimination involves two mechanisms dependent on Apolipoprotein E (ApoE) and cholesterol 24-hydroxylase (CYP46). The aim of this study was to evaluate an intronic variation in CYP46 (intron 2, T --> C ) along with ApoE genotype as risk factors for AD and to establish the correlation between CYP46/ApoE polymorphism and disease progression. One-hundred and fifty-seven AD patients, who had been followed periodically through 1-year follow-up after enrollment, and 134 age- and gender-matched controls entered the study. The distribution of CYP46 genotypes was significantly different in AD compared to controls (P<0.004), being CYP*C allele higher in AD patients ( P<0.002). ApoE 4 genotype was more frequent in AD (41.4%) than in controls (15.9%, P<0.0001). The odds ratio (OR) for AD risk in CYP46*C carriers was 2.8, and in ApoE epsilon4 carriers was 4.05; the OR for having both CYP46*C and ApoE epsilon4 was 17.75, demonstrating the their synergic effect on AD risk. In AD patients, CYP46*C along with ApoE epsilon4 genotype were associated with a higher cognitive decline at 1-year follow-up (P<0.02). These findings provide direct evidence that CYP46 and ApoE polymorphisms synergically increase the risk for AD development, and influence on the rate of cognitive decline

Predictors of progression of cognitive decline in Alzheimer’s disease: the role of vascular and sociodemographic factors

Rates of disease progression differ among patients with Alzheimer’s disease, but little is known about prognostic predictors. The aim of the study was to assess whether sociodemographic factors, disease severity and duration, and vascular factors are prognostic predictors of cognitive decline in Alzheimer’s disease progression. We conducted a longitudinal clinical study in a specialized clinical unit for the diagnosis and treatment of dementia in Rome, Italy. A total of 154 persons with mild to moderate Alzheimer’s disease consecutively admitted to the dementia unit were included. All patients underwent extensive clinical examination by a physician at admittance and all follow-ups. We evaluated the time-dependent probability of a worsening in cognitive performance corresponding to a 5-point decrease in Mini-Mental State Examination (MMSE) score. Survival analysis was used to analyze risk of faster disease progression in relation to age, education, severity and duration of the disease, family history of dementia, hypertension, hypercholesterolemia, and type 2 diabetes. Younger and more educated persons were more likely to have faster Alzheimer’s disease progression. Vascular factors such as hypertension and hypercholesterolemia were not found to be significantly associated with disease progression. However, patients with diabetes had a 65% reduced risk of fast cognitive decline compared to Alzheimer patients without diabetes. Sociodemographic factors and diabetes predict disease progression in Alzheimer’s disease. Our findings suggest a slower disease progression in Alzheimer’s patients with diabetes. If confirmed, this result will contribute new insights into Alzheimer’s disease pathogenesis and lead to relevant suggestions for disease treatment

Competing risks and prognostic stages of cirrhosis: A 25-year inception cohort study of 494 patients

Background Morphological, haemodynamic and clinical stages of cirrhosis have been proposed, although no definite staging system is yet accepted for clinical practice. Aim To investigate whether clinical complications of cirrhosis may define different prognostic disease stages. Methods Analysis of the database from a prospective inception cohort of 494 patients. Decompensation was defined by ascites, bleeding, jaundice or encephalopathy. Explored potential prognostic stages: 1, compensated cirrhosis without oesophago-gastric varices; 2, compensated cirrhosis with varices; 3, bleeding without other complications; 4, first nonbleeding decompensation; 5, any second decompensating event. Patient flow across stages was assessed by a competing risks analysis. Results Major patient characteristics were: 199 females, 295 males, 404 HCV+, 377 compensated, 117 decompensated cirrhosis. The mean follow-up was 145 ± 109 months without dropouts. Major events: 380 deaths, 326 oesophago-gastric varices, 283 ascites, 158 bleeding, 146 encephalopathy, 113 jaundice, 126 hepatocellular carcinoma and 19 liver transplantation. Patients entering each prognostic stage along the disease course were: 202, stage 1; 216, stage 2; 75 stage 3; 206 stage 4; 213 stage 5. Five-year transition rate towards a different stage, for stages 1-4 was 34.5%, 42%, 65% and 78%, respectively (P < 0.0001); 5-year mortality for stages 1-5 was 1.5%, 10%, 20%, 30% and 88% respectively (P < 0.0001). An exploratory analysis showed that this patient stratification may configure a prognostic system independent of the Child-Pugh score, Model for End Stage Liver Disease and comorbidity. Conclusion The development of oesophago-gastric varices and decompensating events in cirrhosis identify five prognostic stages with significantly increasing mortality risks

Predicting disease progression in alzheimer's disease: The role of neuropsychiatric syndromes on functional and cognitive decline

Patients with Alzheimer's disease (AD) have heterogeneous rates of disease progression. The aim of the current study is to investigate whether neuropsychiatric disturbances predict cognitive and functional disease progression in AD, according to failure theory. We longitudinally examined 177 memory-clinic AD outpatients (mean age = 73.1, SD = 8.1; 70.6% women). Neuropsychiatric disturbances at baseline were categorized into five syndromes. Patients were followed for up to two years to detect rapid disease progression defined as a loss of >= 1 abilities in Activities of Daily living (ADL) or a drop of >= 5 points on Mini-Mental State Examination (MMSE). Hazard ratios (HR) were calculated with Gompertz regression, adjusting for sociodemographics, baseline cognitive and functional status, and somatic comorbidities. Most patients (74.6%) exhibited one or more neuropsychiatric syndromes at baseline. The most common neuropsychiatric syndrome was Apathy (63.8%), followed by Affective (37.3%), Psychomotor (8.5%), Manic (7.9%), and Psychotic (5.6%) syndromes. The variance between the observed (Kaplen Meier) and predicted (Gompertz) decline for disease progression in cognition (0.30, CI = 0.26-0.35), was higher than the variance seen for functional decline (0.22, CI = 0.18-0.26). After multiple adjustment, patients with the Affective syndrome had an increased risk of functional decline (HR = 2.0; CI = 1.1-3.6), whereas the risk of cognitive decline was associated with the Manic (HR = 3.2, CI = 1.3-7.5) syndrome. In conclusion, specific neuropsychiatric syndromes are associated with functional and cognitive decline during the progression of AD, which may help with the long-term planning of care and treatment. These results highlight the importance of incorporating a thorough psychiatric examination in the evaluation of AD patients